Advance lane lines detection project¶
Steps of this project are the following:
- Compute the camera calibration matrix and distortion coefficients given a set of chessboard images. Apply a distortion correction to raw images.
- Apply a perspective transform to rectify binary image ("birds-eye view").
- Use color transforms, gradients, etc., to create a thresholded binary image.
- Detect lane pixels and fit to find the lane boundary.
- Determine the curvature of the lane and vehicle position with respect to center.
- Warp the detected lane boundaries back onto the original image.
- Output visual display of the lane boundaries and numerical estimation of lane curvature and vehicle position.
Imports and helper functions¶
from moviepy.editor import VideoFileClip
from IPython.display import HTML
import matplotlib.image as mpimg
import numpy as np
import matplotlib.pyplot as plt
import cv2
import math
import glob
from IPython import display
%matplotlib inline
# %matplotlib qt
# display an image with title
def print_image(image, title = '', grayscale = False):
fig = plt.figure()
if not (title == ''):
plt.suptitle(title + ':')
if (grayscale):
plt.imshow(image.squeeze(), cmap = "gray")
else:
plt.imshow(image.squeeze())
plt.show()
# display two images
def print_two_images(img1, img2, title1 = '', title2 = '', grayscale1 = False, grayscale2 = False):
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(24, 9))
f.tight_layout()
if (grayscale1):
ax1.imshow(img1, cmap = "gray")
else:
ax1.imshow(img1)
ax1.set_title(title1, fontsize=30)
if (grayscale2):
ax2.imshow(img2, cmap = "gray")
else:
ax2.imshow(img2)
ax2.set_title(title2, fontsize=30)
plt.subplots_adjust(left=0., right=1, top=0.9, bottom=0.)
Processing functions:¶
Image undistortion¶
As the first step of the image processing pipeline we need to undistort the image to remove camera distortion.
1. Calibrate camera¶
First of all, we need to calibrate the camera basing on images of the chess board taken on the flat surface. We will calculate image and object points - coordinates of inner corners of the chess board on the image and normalized coordinates.
I start by preparing "object points", which will be the (x, y, z) coordinates of the chessboard corners in the world. Here I am assuming the chessboard is fixed on the (x, y) plane at z=0, such that the object points are the same for each calibration image. Thus, objp is just a replicated array of coordinates, and objpoints will be appended with a copy of it every time I successfully detect all chessboard corners in a test image. imgpoints will be appended with the (x, y) pixel position of each of the corners in the image plane with each successful chessboard detection.
I then used the output objpoints and imgpoints to compute the camera calibration and distortion coefficients using the cv2.calibrateCamera() function.
# function for creating of object points and image points. Takes photo paths with chess board and size of
# the board (amount of inner corners)
def calibrate_camera(image_files_mask = './camera_cal/calibration*.jpg', x_chess_size = 9, y_chess_size=6):
images = glob.glob(image_files_mask)
objpoints = []
imgpoints = []
# get, let z=0
objp = np.zeros((x_chess_size*y_chess_size,3), np.float32)
objp[:,:2] = np.mgrid[0:x_chess_size,0:y_chess_size].T.reshape(-1,2)
# get chessboard corners for every image
for file in images:
img = mpimg.imread(file)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, corners = cv2.findChessboardCorners(gray, (x_chess_size, y_chess_size), None)
if ret == True:
imgpoints.append(corners)
objpoints.append(objp)
print("Final points size: {0}, {1}".format(len(objpoints), len(imgpoints)))
return (objpoints, imgpoints)
2. Undistort image¶
def cal_undistort(img, objpoints, imgpoints):
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, img.shape[1::-1], None, None)
undist = cv2.undistort(img, mtx, dist, None, mtx)
return undist
3. Test¶
Lets see how good calculated image points are on calibration and test images from the car.
(objpoints, imgpoints) = calibrate_camera()
images = glob.glob('./camera_cal/calibration*.jpg')
for image in images:
img = mpimg.imread(image)
res = cal_undistort(img, objpoints, imgpoints)
print_two_images(img, res, "raw", "undistorted")
